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ABSTRACT 
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COEA,  concludes  that  five  techniques  are  particularly  worthy  of  COEA 
practitioner  consideration,  and  then  illustrates  the  use  of  two 
techniques — the  Analytic  Hierarchy  Process  (AHP)  and  Multiattribute 
Utility  Theory  (MAUT) . 
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Multiattribute  Methodologies  for 
Decision  Making  in  COEAs 


1.  Purpose.  This  report  documents  the  Multiattribute 
Methodologies  for  Decision  Making  in  COEAs  study.  The  study  was 
conducted  in  partnership  between  TRAC-FBHN  and  TRAC-WSMR  and 
sponsored  by  the  office  of  the  Director  of  MANPRINT,  HQ  DA. 

2.  References.  See  Appendix  A. 

3.  Terms  of  Reference. 

a.  Background.  This  study  discusses  and  illustrates  the  use 
of  well  accepted  operations  research  techniques  for  integrating 
multiple  decision  attributes,  often  conflicting,  within  the  context 
of  Cost  and  Operational  Effectiveness  Analyses (COEA) .  As  an 
oversimplified  example,  consider  a  case  in  which  the  Army  is  going 
to  buy  one  of  three  alternative  helicopters.  The  decision  as  to 
which  one  to  buy  may  depend  on  any  number  of  attributes. 

Attributes  are  the  criteria  decision  makers  consider  in  making 
choices.  In  COEA  they  might  include  cost,  combat  effectiveness, 
supportability,  manpower,  safety,  and  the  like.  Typically  COEA 
focus  on  the  assessment  of  the  alternatives  with  regard  to 
specified  attributes.  The  emphasis  is  on  estimating  cost,  combat 
effectiveness,  supportability,  manpower,  safety,  and  so  on.  There 
is  often  little  or  no  attempt  to  integrate  the  attribute  estimates 
or  outcomes  into  an  overall  scheme  to  further  guide  the  decision 
maker  in  the  choice  of  alternatives. 

b.  Problem.  There  is  presently  a  lack  of  well  accepted 
techniques  within  the  Army  analytic  community  for  integration  of 
distinct  and  often  conflicting  attributes  within  COEA  and  other 
studies.  When  analysis  delineates  alternatives  and  leads  to 
recommendations  to  Army  decision  makers,  such  analysis  should  be 
consistent  with  state  of  the  art  operations  research  techniques  so 
as  to  be  understandable,  verifiable,  repeatable,  and  robust  to 
criticism  from  members  of  the  analytic  community.  Present  Army 
analysis  with  respect  to  integration  of  multiple  attributes 
frequently  fails  to  meet  these  standards. 

c.  Impact  of  Problem.  The  disuse  of  well  accepted,  practical 
integration  techniques  makes  trade-offs  among  multiple  attributes 
awkward  and  sometimes  controversial.  Trade-offs  may  often  defy  a 
ready  understanding  by  the  target  audience  or  the  decision  maker. 

d.  Objectives.  This  study  has  the  following  two  objectives: 

(1)  To  delineate  and  summarize  practical  multiattribute 
integration  techniques  for  use  in  COEA. 

(2)  To  demonstrate  the  use  of  two  of  the  most  promising 
multiattribute  techniques  in  COEA. 


e.  Essential  Elements  of  Analysis  (EEA) .  This  study  has  the 
following  EEA: 

EEA  1.  What  are  the  principal  multiattribute  techniques  for 
integrating  attributes  in  the  choice  of  alternatives? 

EEA  2.  What  are  the  most  practical  multiattribute 
techniques  for  use  in  COEA? 

EEA  3.  How  do  the  most  practical  techniques  compare  in  a 
typical  COEA  application? 

f.  Scope.  This  study  considers  the  following  decision 
situation.  There  are  a  small  number  of  alternatives,  say  no  more 
than  30,  which  are  to  be  compared  with  regard  to  a  small  number  of 
attributes,  also  say  no  more  than  30.  Outcomes  for  the  alternatives 
with  regard  to  the  attributes  have  been  estimated.  Techniques  under 
consideration  are  designed  to  integrate  the  various  estimated 
outcomes  into  an  overall  ranking  of  the  alternatives. 

(1)  Integration  techniques  under  consideration  are  referred 
to  in  the  literature  with  the  terms  multiattribute  (multiple 
attribute) ,  multiobjective  (multiple  objective) ,  multidimensional 
(multiple  dimensional) ,  and  multicriteria  (multiple  criteria) .  The 
term  used  herein  is  multiattribute. 

(2)  Multiattribute  integration  techniques  are  distinguished 
from  optimization  techniques,  such  as  linear  programming,  integer 
programming,  or  goal  programming.  Multiattribute  techniques  focus 
on  the  determination  of  the  integration  process,  the  optimization 
process  being  trivial.  In  programming  the  determination  of  the 
integration  process,  the  objective  function,  is  secondary  to  the 
nontrivial  optimization  over  an  infinite  or  at  an  least  extremely 
large  number  of  alternatives. 

g.  Limitations. 

(1)  Multiattribute  techniques  are  often  roughly  categorized 
as  either  descriptive  or  prescriptive.  Descriptive  techniques 
derive  from  research  into  behavioral  decision  theory  which  seeks 
primarily  to  describe  how  people  make  decisions.  Prescriptive 
techniques,  the  concern  of  operations  research,  seek  to  help  people 
make  better  decisions.  Prescriptive  techniques  are  founded  on 
compelling  axioms  or  practical  decision  criteria  whereas  descriptive 
techniques  rely  on  empirical  decision  data.  This  study  focuses  on 
prescriptive  techniques.  Consequently,  techniques  such  as  a 
regression  model  based  on  historical  decision  data.  Elimination  by 
Aspects  (EBA) ,  Linear  Programming  Techniques  for  Multidimensional 
Analysis  of  Preference  (LINMAP) ,  and  Maximum  Likelihood  Hierarchical 
(MLH)  are  not  considered. 

(2)  While  the  operations  research  literature  abounds  with 
theoretically  sound  or  practical  techniques  for  the  individual 
decision  maker,  the  same  cannot  be  said  for  the  decision  making 
group.  Techniques  considered  in  this  study  were  designed  primarily 
for  use  by  a  single  decision  maker.  However,  some,  such  as  the 
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Analytic  Hierarchy  Process  (AHP) ,  may  be  highly  adaptable  to  a 
decision  making  group. 

(3)  Of  the  techniques  identified  in  the  compendium, 
several  are  selected  as  appropriate  for  general  use  in  COEA. 

Factors  used  to  determine  appropriateness  include  theoretical 
soundness,  applicability  to  the  COEA  process,  ease  of  use, 
availability  of  application  software,  data  requirements,  and 
prevalence  (which  encompasses  demonstrated  effectiveness) .  The 
criteria  serve  as  a  general  screen  for  techniques.  They  are  not 
independent,  nor  are  they  necessarily  criteria  that  others  may 
invoke.  They  do  strike  us  as  reasonable  for  present 
circumstances.  The  purpose  of  this  paper  is  not  to  evaluate 
explicitly  the  techniques  against  the  criteria,  but  rather  to  set 
out  viable  techniques  as  well  as  some  illustration  of  their  use. 
Nonetheless,  for  the  sake  of  clarity,  we  offer  the  following 
explanation  of  our  criteria: 

By  theoretical  soundness,  we  mean  the  extent  to  which  a 
technique  has  successfully  withstood  scrutiny  in  refereed  journals. 

By  applicability  to  the  COEA  process,  we  mean  the  apparent 
match  between  what  a  technique  purports  to  accomplish  and  the 
purpose  of  a  COEA  (i.e.,  "to  select  the  preferred  alternative"). 

By  ease  of  use,  we  mean  the  degree  of  difficulty  involved  in 
gathering  required  data,  employing  software,  explicating  outcomes, 
and  the  like. 

By  availability  of  application  software,  we  mean  ready 
accessibility  of  electronic  aids  for  performing  a  technique  (which 
m.’ght  be  tedious  if  performed  by  hand)  . 

By  data  requirements,  we  mean  the  extent  to  which  a  technique 
calls  for  data  that  may  be  difficult  to  obtain  or  process. 

By  prevalence,  we  mean  an  evolving  history  of  use  in  study 
applications  and/or  appearance  of  the  technique  is  refereed 
journals. 

Of  the  acceptable  techniques  several  are  illustrated  on  COEA  as 
permitted  by  the  study  resource  constraints. 

h.  Assumptions.  Multiattribute  techniques  enable  trade-offs 
which  are  consistent  with  decision  makers'  preferences.  To 
accomplish  this,  the  techniques  often  rely  on  data  directly  from 
the  decision  makers.  One  must  assume  that  decision  makers  will  be 
available  to  provide  data  as  necessary. 

i.  Constraints.  Techniques  are  demonstrated  on  a  recently 
completed  COEA.  Because  the  demonstrations  are  ancillary  to  and 
separate  from  the  primary  COEA  effort,  data  required  by  the 
techniques  may  not  be  readily  available.  Given  such  a  situation 
and  a  desire  to  use  the  techniques,  parametric  analysis  or  output 
to  input  analysis  may  be  conducted.  Finally,  no  criticism  of 
analytical  procedure  in  the  selected  COEA  is  made  or  implied. 
Resources  are  constrained  to  one  professional  staff  year  to  be 
expended  in  no  more  than  one  calendar  year. 
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j.  Alternative  Techniques. 

(1)  Table  1  list  the  multiattribute  techniques  considered 
in  this  study. 


Table  1 

Multiattribute  Techniques 

Dominance 

Maximin 

Maximax 

Majority  Rule 

Koler's  Ranking  Technique 

Conjunctive  Technique 

Disjunctive  Technique 

Stochastic  Dominance 

Lexicographic 

Lexicographic  with  Minima 

Key  Attribute 

ELECTRE 

Permutation 

Analytic  Hierarchy  Process  (AHP) 

Multiattribute  Value  Theory  (MAVT) 

Simple  Additive  Weighting  (SAW) 

Technique  for  Order  Preference  by 
Similarity  to  Ideal  Solution  (TOPSIS) 
Cost-effectiveness  ratios 
Multiattribute  Utility  Theory  (MAUT) 

These  techniques  are  summarized  in  Appendix  B.  In  accordance  with 
the  limitations  of  the  study  the  techniques  listed  in  Table  2  are 
found  appropriate  for  general  use  in  COEA.  Note  that  the  term 
"theory"  herein  connotes  a  technique. 


Table  2 

Techniques  Appropriate  for  General 
Use  in  COEA 

Dominance  (including  stochastic) 
Conjunctive  technique 
Analytic  Hierarchy  Process  (AHP) 
Multiattribute  Value  theory  (MAVT) 
Multiattribute  Utility  theory  (MAUT) 


The  fact  that  a  technique  is  not  found  appropriate  for  general  use 
does  not  mean  that  the  technique  has  no  worth.  In  particular 
simple  ranking  techniques  such  as  majority  rule  and  Koler's  ranking 
technique  have  great  potential  for  use  in  COEAs.  However,  further 
research  is  required  for  a  full  understanding  for  their  application 
in  a  COEA  environment.  On  the  other  hand,  techniques  such  as  cost- 
effectiveness  ratios  or  TOPSIS,  are  not  considered  appropriate  for 
general  use  in  COEAs  because  of  their  unique  underlying 
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assumptions.  If  analysis  shows  that  these  unique  assumptions  hold 
for  a  particular  application,  then  their  use  should  be  considered. 

(2)  Because  of  its  rationality  and  ease  of  use,  dominance 
should  always  be  applied  in  a  COEA.  However,  it  is  most  likely 
that  dominated  alternatives  have  fallen  out  of  consideration  before 
the  COEA.  Practically,  this  technique  would  serve  only  as  a 
screening  device  to  determine  acceptable  (non-dominated)  and 
unacceptable  (dominated)  alternatives.  If  true  minimum  acceptable 
levels  have  been  set,  e.g.,  as  in  an  Operational  Requirements 
Document  (ORD) ,  the  conjunctive  technique  should  also  be  applied  to 
screen  out  any  unacceptable  alternatives. 

(3)  From  the  above  list,  we  demonstrate  AHP  and  MAUT. 

Since  analysts  typically  view  utility  theory  as  the  stochastic 
analogue  of  value  theory,  and  since  resource  constraints  preclude 
the  demonstration  of  both  value  theory  and  utility  theory,  utility 
theory  (MAUT)  was  selected  since  it  is  the  more  general  of  the  two. 

4.  Structuring  the  Decision  Problem. 

a.  Defining  the  Problem. 

(1)  The  multiattribute  techniques  are  demonstrated  in 
conjunction  with  recent  TOW  Sight  Improvement  Program  (TSIP) 
Abbreviated  Cost  and  Operational  Effectiveness  Analysis  (COEA) . 

This  study  was  conducted  to  scrutinize  alternative  antiarmor 
capabilities  in  mechanized  infantry  and  light  infantry  units.  The 
demonstration  will  consider  only  a  subset  of  alternatives  from  the 
mechanized  infantry  portion  of  the  COEA.  The  mainstay  of  the 
current  antiarmor  capability  of  these  units  is  the  tube-launched, 
optically-tracked,  wire-guided  (TOW)  missile.  This  missile  will  be 
upgraded  to  the  TOW  2B  configuration.  In  addition,  the  TOW  sight 
improvement  program  seeks  to  replace  the  current  sight  with  a  new 
sight. 


(2)  Mechanized  infantry  battalions  include  four  mechanized 
infantry  companies  fielding  Bradley  Fighting  Vehicles  (BFV)  firing 
TOW  missiles.  In  addition,  the  battalion  contains  an  antiarmor 
company  with  improved  TOW  vehicles  (ITV) ,  also  firing  TOW. 

b.  Alternatives.  The  infantry  antiarmor  COEA  considered 
improving  either  the  sight  of  the  ITV  TOW  or  the  sight  of  the  BFV 
TOW  or  sights  of  both.  Table  3  lists  the  four  demonstration 
alternatives . 
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Table  3 
Alternatives 


Mech  Inf  Co 
BFV  TOW  Sight 


Anti armor  Co 

ITV  TOW  Sight  Alternative 


Base  Case 

Current 

Current 

Alt  1 

Current 

New 

Alt  2 

New 

Current 

Alt  3 

New 

New 

c.  Attributes.  The  development  of  study  attributes  begins 
with  the  specification  of  the  following  very  broad  decision 
criterion:  "to  efficiently  provide  an  infantry  antiarmor 

capability."  Working  from  this  broad  criterion,  a  number  of  more 
specific  decision  criteria  were  delineated.  A  preliminary  list  of 
these  criteria  appear  in  Table  C-l  of  Appendix  C.  This  process  of 
refinement  continued  until  a  set  of  quantifiable  attributes  was 
developed.  These  attributes,  which  are  intended  to  capture  as 
completely  as  possible  all  decision  criteria,  are  broadly 
categorized  as  benefit  attributes  or  resource  attributes.  These 
attributes  are  listed  in  Table  4.  There  is  a  single  resource  or 
cost  attribute  which  is  a  summation  of  five  components.  These 
parallel  the  five  components  typically  found  in  COEA  cost 
analysis.  The  scale  for  this  attribute  is  FY92  constant  dollars 
with  a  twenty  year  time  horizon.  For  purposes  of  this  study,  the 
sustainment  component  includes  manpower,  personnel,  and  training 
resource  impacts. 


Table  4 
Attributes 


Benefit  attributes: 

1.  Ratio  of  red  losses  to  blue  losses  for  European 
brigade  meeting  engagement 

2.  Ratio  of  red  losses  to  blue  losses  for  European 
balance  task  force  defense 

3.  Ratio  of  red  losses  to  blue  losses  for  Southwest 
Asian  brigade  meeting  engagement 

Resource  attribute  (summation  of  five  components) 

a.  Development  Costs 

b.  Production  Costs 

c.  Military  Construction  Costs 

d.  Fielding  Costs 

e.  Sustainment  Costs  (including  Manpower) 
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(1)  Some  decision  criteria  could  be  treated  in  either  a 
benefit  (output)  or  resource  (cost,  input)  fashion.  For  example, 
consider  air  transportation  as  a  factor.  COEA  alternatives  may  be 
such  that  some  are  air  transportable  and  some  are  not. 

Consequently  the  binary  attribute,  "air  transportable"  (yes  or  no) , 
could  be  treated  as  a  benefit  attribute.  On  the  other  hand,  air 
transportability  might  be  a  requirement  specified  in  the  system 
ORD.  In  this  case,  air  transportability  could  be  captured  in 
increased  development,  production,  or  supportability  costs  of  the 
alternatives.  For  the  purposes  of  the  demonstration,  all 
sustainment  or  supportability  and  MANPRINT  criteria  are  handled  on 
the  resource  or  cost  side.  For  example,  system  safety  is  assumed 
specified  at  a  certain  level,  with  differences  among  alternatives 
possibly  showing  as  differences  in  development,  production,  or 
sustainment. 

(2)  In  developing  attributes,  the  priority  of  v'-.he  criteria 
and  the  comprehensiveness  of  the  assessment  modeling  must  be 
considered.  For  example,  consider  the  criteria  "combat 
effectiveness"  and  "equipment  availability."  Availability  alone  is 
not  important.  It  is  important  only  through  its  influence  on 
combat  effectiveness.  Combat  effectiveness  may  be  assessed  with  a 
combat  model  that  is  capable  of  representing  the  effect  of 
alternatives  with  differing  availability  rates.  Thus,  availability 
may  be  subsumed  under  the  combat  effectiveness  attributes. 

(3)  The  benefits  of  alternatives  are  assessed  using 
attrition  measures  from  a  high  resolution  combat  simulation  model. 
Three  different  combat  scenarios  are  represented.  For  each 
scenario  the  loss  exchange  ratio  (LER) ,  red  losses  divided  by  blue 
losses,  is  an  attribute.  The  three  scenarios,  "European  brigade 
meeting  engagement,"  "European  balanced  task  force  defense,"  and 
"Southwest  Asian  (SWA)  brigade  meeting  engagement,"  pit  1996 
projected  blue  forces  against  2004  projected  red  forces. 
Descriptions  of  the  scenarios  appear  in  Appendix  C.  The  fact  that 
the  attributes  are  based  on  only  three  scenarios  does  not  mean  that 
conflict  is  contemplated  in  only  these  three  narrowly  defined 
situations.  Rather,  these  scenarios  are  viewed  as  representative 
of  conflicts  that  might  actually  occur. 

5.  Demonstration  of  Analytic  Hierarchy  Process 

a.  For  the  demonstration  of  AHP  Figure  1  illustrates  the  COEA 
attributes  in  a  hierarchical  fashion. 
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Efficiently  Provide  an  Infantry 
Antiarmor  Capability 


COST 


EFFECTIVE  ANTI ARMOR 
CAPABILITY 


BC  ALT  ALT  ALT 
12  3 


EUR  MGT 
LER 


BC  ALT  ALT  ALT 
12  3 


EUR  DEF 
LER 


BC  ALT  ALT  ALT 
12  3 


SWA  MGT 
LER 


BC  ALT  ALT  ALT 
12  3 


Figure  1.  The  Attribute  Hierarchy 

The  hierarchy  requires  six  distinct  scalings.  At  the  top  the 
decision  maker  provides  scale  values  on  the  relative  importance  of 
cost  ana  effectiveness  in  meeting  the  overall  decision  criterion. 
Continuing  down  the  hierarchy,  the  decision  maker  scales  the 
relative  importance  of  the  three  LER  attributes  in  terms  of  their 
contribution  to  the  effectiveness  attribute.  Next  the  alternatives 
are  scaled  in  terms  of  their  contribution  to  the  cost  attribute, 
the  European  meeting  LER  attribute,  the  European  Defense  LER 
attribute,  and  the  SWA  meeting  LER  attribute. 

b.  For  demonstration  purposes  hypothetical  scale  values  are 
used.  While  we  feel  that  these  are  realistic  values,  we  do  not 
claim  empirical  validity.  The  AHP  scale  values  are  defined  in 
Table  2  of  Appendix  B.  The  eigenvector  weights  of  the  respective 
scale  value  matrices  are  computed  using  a  FORTRAN  subroutine  from 
the  IMSL  library  for  mathematical  applications. 

(1)  Table  5  gives  the  scale  values  for  the  relative 
importance  of  the  contribution  of  cost  and  antiarmor  capability  to 
the  decision.  This  choice  of  values  weights  erst  and  effectiveness 
equally. 


Table  5 

Pairwise  Comparisons  of 
Cost  and  Antiarmor  Capability 

Antiarmor 

Cost  Capability 

Cost  1  1 

Antiarmor 

Capability  1  1 

Eigenvector  weightings  -  (0.50,  0.50) 
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(2)  Table  6  gives  the  scale  values  for  the  relative 
importance  of  the  three  LER  attributes  in  their  contribution  to  the 
antiarmor  capability.  The  scale  values  have  been  chosen  so  that 
the  first  two  LER  attributes  are  equally  valued  and  the  third  is 
weakly  more  important. 


Table  6 

Pairwise  Comparisons  of 
LER  Attributes 


EUR  MGT 
LER 

EUR  DEF 
LER 

SWA  MGT 
LER 


EUR  MGT 
LER 

1 

1 

3 


EUR  DEF 
LER 

1 

1 

3 


SWA  MGT 
LER 

1/3 

1/3 

1 


Principal  Eigenvalue  -  3.00 

Eigenvector  weighting  -  (0.20,  0.20,  0.60) 


A  principal  eigenvalue  of  3.00  indicates  that  the  scale  values  are 
completely  consistent. 


(3)  Table  7  gives  the  scale  values  for  the  cost  of  the 
alternatives.  These  scale  values  are  derived  from  cost  estimates 
(in  FY92  constant  million  dollars)  presented  in  Table  C-2  of 
Appendix  C.  The  scale  values  have  been  chosen  so  that  the  base 
case  is  valued  strongly  over  alternatives  one  and  two,  alternatives 
one  and  two  are  equally  valued,  and  alternatives  one  and  two  are 
valued  strongly  over  alternative  three.  Irt  addition  the  base  case 
is  valued  absolutely  over  alternative  three.  The  "cheaper  is 
better"  principle  applies. 


Table  7 

Pairwise  Comparisons  of 
Cost  of  Alternatives 


Base 

Case 

Alt 

1 

Alt 

2 

Alt 

3 

Cost 

350 

900 

982 

1344 

Base  Case 

1 

5 

5 

9 

Alt  1 

1/5 

1 

1 

5 

Alt  2 

1/5 

1 

1 

5 

Alt  3 

1/9 

1/5 

1/5 

1 

Principal  Eigenvalue  -  4.13 

Eigenvector  weighting  -  (0.64,  0.16,  0.16,  0.04) 
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A  principal  eigenvalue  of  4.13  indicates  that  the  scale  values  are 
highly  consistent. 

(4)  Tables  8,  9,  and  10  give  scale  values  of  the 
alternatives  for  the  LER  attributes.  Also  these  tables  give  the 
eigenvector  weightings  and  principal  eigenvalues.  These  scale 
values  are  based  on  the  median  LER  estimates  from  Tables  C-6,  C-7, 
and  C-8  of  Appendix  C.  In  Table  8  the  base  case  and  alternative 
one  are  valued  the  same.  Alternative  two  is  valued  strongly  over 
the  base  case  and  alternative  one.  Alternative  three  is  valued 
strongly  over  alternative  two.  Also  alternative  three  is  valued 
absolutely  over  the  base  case  and  alternative  one.  In  contrast  to 
cost,  higher  LERs  are  "better." 


Table  8 

Pairwise  Comparisons  of 
European  Meeting  LER  Estimates 


Base 

Case 

Median  LER  1.00 

Base  Case  1 

Alt  1  1 

Alt  2  5 

Alt  3  9 


Alt 

Alt 

Alt 

1 

2 

3 

1.04 

1.40 

1.77 

1 

1/5 

1/9 

1 

1/5 

1/9 

5 

1 

1/5 

9 

5 

1 

Principal  Eigenvalue  -  4.13 

Eigenvector  weighting  -  (0.06,  0.06,  0.22,  0.66) 

A  principal  eigenvalue  of  4.13  indicates  that  the  scale  values  are 
highly  consistent. 


In  Table  9  alternatives  one  and  two  are  equally  valued. 
Alternatives  one  and  two  are  valued  weakly  over  the  base  case  and 
alternative  three  is  value  weakly  over  alternatives  one  and  two. 
Also  alternative  three  is  valued  strongly  over  the  base  case. 
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Table  9 

Pairwise  Comparisons  of 


European 

Defense 

LER 

Estimates 

Base 

Alt 

Alt 

Alt 

Case 

1 

2 

3 

Median  LER 

2.29 

2.44 

2.38 

2.57 

Base  Case 

1 

1/3 

1/3 

1/5 

Alt  1 

3 

1 

1 

1/3 

Alt  2 

3 

1 

1 

1/3 

Alt  3 

5 

3 

3 

1 

Principal  Eigenvalue  -  4.04 

Eigenvector  weighting  -  (0.08,  0.20,  0.20,  0.52) 

A  principal  eigenvalue  of  4.04  indicates  that  the  scale  values  are 
highly  consistent. 

In  Table  10  the  base  case  and  alternative  one  are  equally  valued. 
Alternative  two  is  valued  weakly  over  the  base  case  and  alternative 
one.  Alternative  three  is  valued  weakly  over  alternative  two  and 
strongly  over  the  base  case  and  alternative  one. 

Table  10 

Pairwise  Comparisons  of 
SWA  Meeting  LER  Estimates 


Base 

Case 

Alt 

1 

Alt 

2 

Alt 

3 

Median  LER  1 

.34 

1.37 

1.49 

1.62 

Base  Case 

1 

1 

1/3 

1/5 

Alt  1 

1 

1 

1/3 

1/5 

Alt  2 

3 

3 

1 

1/3 

Alt  3 

5 

5 

3 

1 

Principal  Eigenvalue 
Eigenvector  weighting 

-4.04 
-  (0. 

10,  0.10, 

0.24, 

0.56) 

principal  eigenvalue  of 

4.04 

indicates 

that 

the  scale 

highly  consistent. 

d.  The  final  step  in  the  AHP  is  to  aggregate  the  separate 
weightings  into  an  overall  weighting  for  each  alternative.  The 
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following  linear  format,  described  in  Appendix  B,  is  used. 

AHP  Weighting  =  0.5  *  (Cost  Weighting) 

+  0.5  *  0.2  *  (EUR  MGT  LER  Weighting) 

+  0.5  *  0.2  *  (EUR  DEF  LER  Weighting) 

+  0.5  *  0.6  *  (SWA  MGT  LER  Weighting) 

Table  11  gives  the  overall  AHP  weighting  for  each  alternative. 


Table  11 

AHP  Weightings  of  Alternatives 


Alternative 
Base  Case 
Alt  1 
Alt  2 
Alt  3 


Weighting 

0.36 

0.13 

0.20 

0.31 


The  base  case  ranks  first  followed  by  alternative  three.  The 
greater  effectiveness  of  alternative  three  is  offset  by  its  greater 
cost.  This  results  in  part  from  valuing  the  importance  of  cost  and 
effectiveness  equally.  Had  effectiveness  been  valued  just  very 
weakly  over  cost  the  scale  values  of  Table  5  would  be  replaced  by 
those  of  Table  12  below. 


Table  12 

Revised  Pairwise  Comparisons  of 
Cost  and  Antiarmor  Capability 

Antiarmor 

Cost  Capability 

Cost  1  1/2 

Anti armor 

Capability  2  1 

Eigenvector  weightings  -  (1/3,  2/3) 

With  these  new  weightings  for  cost  and  effectiveness  the  overall 
weightings  of  the  alternatives  become  those  of  Table  13. 
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Table  13 

Revised  AHP  Weightings  of  Alternatives 


Alternative 
Base  Case 
Alt  1 
Alt  2 
Alt  3 


Weighting 

0.27 

0.12 

0.21 

0.40 


In  this  case  alternative  three  ranks  first  followed  by  the  base 
case.  One  can  show  with  a  simple  computation  that  if  effectiveness 
is  valued  at  1.21  over  cost,  the  base  case  and  alternative  three 
tie  for  the  first  rank  with  equal  weightings. 

6.  Demonstration  of  Multiattribute  Utility  Theory 

a.  As  with  AHP  the  application  of  MAUT  begins  with 
hierarchical  decomposition  illustrated  in  Figure  1.  The  complete 
demonstration  requires  three  steps:  modeling  utility,  modeling  the 
probability  distribution  of  outcomes,  and  integrating  the  two 
models. 


b.  Modeling  utility.  To  simplify  the  demonstration,  the 
attributes  are  assumed  to  satisfy  appropriate  utility  independence 
conditions  so  that  the  multiattribute  utility  function  has  either 
the  additive  or  the  multiplicative  form  of  the  multilinear 
decomposition  (refer  to  Appendix  B  Compendium) .  Formal  checking  of 
this  assumption  would  be  required  in  an  actual  application.  Its 
failure  to  hold  would  require  more  complex  techniques.  Following 
this  assumption  the  first  step  is  to  determine  the  single  attribute 
utility  functions. 

(1)  For  demonstration  purposes  hypothetical  functions  are 
used.  These  functions  are  such  that  the  decision  maker  is  risk 
neutral  with  regard  each  single  attribute.  While  one  hopes  that 
these  are  reasonable  functions,  no  claim  is  made  to  their  empirical 
validity.  Table  14  gives  these  functions  which  are  scaled  from  0 
to  1  over  their  range.  Risk  neutrality  requires  that  these 
functions  be  linear. 
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Table  14 

Single  Attribute  Utilities 


Attribute 

Range 

Constant 

Utility 

Scaling 

Cost 

200  to  1500 

UC  (c) 

=  ( 1500-c) / 1300 

KC 

EUR  MGT 

LER 

0.25  to  4.0 

UM(x) 

=  4 (x  -  0.25) / 15 

KM 

EUR  DEF 

LER 

0.25  to  4.0 

UD(y) 

=  4 (y  -  0 . 25) /15 

KD 

SWA  MGT 

LER 

0.25  to  4.0 

US(Z) 

=  4 ( Z  -  0.25) / 15 

KS 

Using  the  four  scaling  constants  introduced  in  Table  14  the  utility 
function,  U(c,x,y,z),  decomposes  as 

U(c, x, y , z)  =  KC*UC (c)  +  KM*UM(x)  +  KD*UD (y)  +  KS*US(z) 

+  K* (KC*KM*UC(C) *UM(x)  +  KC*KD*UC (c) *UD (y)  + 

KC*KS*UC (c) *US (z)  +  KM*KD*UM(x) *UD(y)  + 
KM*KS*UM(X) *US(Z)  +  KD*KS*UD (y) *US (z) ) 

+  K*K*(KC*KM*KD*UC(c)*UM(x)*UD(y)  + 

KC*KM*KS*UC(C)*UM(X)*US(Z)  + 

KC*KD*KS*UC (c) *UD (y) *US ( z)  + 

KM*KD*KS*UM (X) *UD (y ) *US ( Z ) ) 

+  K*K*K*(KC*KM*KD*KS*UC(c)*UM(x)*UD(y)*US(z) ) 

Over  the  range  of  attribute  values  the  function  U  is  scaled  from  0 
to  1.  The  value  0  is  obtained  when  all  attributes  are  at  their 
worst  value,  i.e.,  U(1500, 0.25, 0.25, 0.25)  =  0.  The  value  1  is 
obtained  when  all  attributes  are  at  their  best  value,  i.e., 

U( 200, 4. 0,4. 0,4.0)  =1.  If  the  constant  K  in  the  decomposition  of 
U  is  0  the  additive  form  holds  otherwise  the  multiplicative  form 
holds. 


(2)  In  the  next  step  values  of  the  four  scaling 
constants,  KC  through  KS,  are  determined.  This  requires  detailed 
and  complex  assessment  sessions  with  the  decision  maker.  Software 
such  as  IDEA  is  useful  in  simplifying  this  assessment.  After  the 
four  scaling  constants,  KC  through  KS,  are  determined,  the  constant 
K  is  determined  numerically  as  the  solution  of  a  polynomial.  Table 
15  gives  hypothetical  constant  values  for  the  demonstration.  The 
scaling  constants,  KC  through  KS,  have  a  utility  interpretation  at 
the  extreme  values  of  the  attributes.  They  are  not  to  be 
interpreted  as  the  relative  importance  of  an  attribute,  i.e.,  KC  is 
not  interpreted  as  the  relative  importance  of  the  cost  attribute. 
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Table  15 

MAUT  Scaling  Constants 


Utility 


•nstant 

Equivalent 

Value 

KC 

U(200, 0.25, 0.25, 0.25) 

0.200 

KM 

U(1500,4.0,0.25,0.25) 

0.300 

KD 

U(1500,0.25,4.0,0.25) 

0.300 

KS 

U(1500, 0.25, 0.25, 4.0) 

0.400 

K 

not  applicable 

-0.409 

c.  Modeling  Outcomes.  To  simplify  the  demonstration  the 
attributes  for  any  particular  alternative  are  assumed  to  be 
probabilistically  independent  (refer  to  Appendix  B  Compendium) . 
Formal  checking  of  this  assumption  would  be  required  in  an  actual 
application.  Its  failure  to  hold  would  require  more  complex 
techniques.  Following  this  assumption  probability  distributions  of 
outcomes  for  each  alternative  and  each  attribute  must  be  developed. 

(1)  Because  of  the  assumptions  of  independence  and  risk 
neutrality  only  the  means  of  the  distributions  of  outcomes  are 
required.  Table  16  summarizes  the  necessary  values  from  Tables 
C-5 ,  C-6,  C-7,  and  C-8  of  Appendix  C. 

Table  16 

Means  of  Outcomes 

Attributes 


Alternative 

Cost 

EUR  MGT 
LER 

EUR  DEF 
LER 

SWA  MGT 
LER 

Base  Case 

350 

1.06 

2.35 

1.30 

Alt  1 

900 

1.08 

2.46 

1.40 

Alt  2 

982 

1.45 

2.36 

1.52 

Alt  3 

1344 

1.81 

2.78 

1.63 

d.  Integration.  The  last  step  is  to  combine  the  utility 
model  and  the  probability  model.  This  step  proceeds  by  computing 
the  expected  utility  for  each  alternative  by  integrating  the 
multivariate  utility  function  with  the  probability  distribution  of 
outcomes.  The  alternatives  are  then  ranked  by  their  expected 
utility.  Because  of  the  assumptions  of  independence  and  risk 
neutrality,  the  expected  utility  of  an  alternative  may  be  computed 
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by  substituting  the 
utility  function  U(c 
for  each  alternative 
basis  for  selection 

appropriate  mean  values  from  Table  16  into  the 
,x,y,z).  Table  17  gives  the  expected  utilities 
.  Expected  utility  represents  a  reasonable 
of  an  alternative  by  a  decision  maker. 

Table  17 

Expected  Utilities 

Alternative 

Expected 

Utility 

Base  Case 

0.483 

Alt  1 

0.428 

Alt  2 

0.447 

Alt  3 

0.465 

As  was  the  situation  with  AHP  the  base  case  ranks  first  followed  by 
alternative  three.  If  the  value  of  KC,  which  equals  U(200,0.25, 
0.25,0.25),  were  decreased  from  0.20  to  0.15,  the  expected 
utilities  of  Table  17  would  be  replaced  by  those  of  Table  18. 

Table  18 

Revised  Expected  Utilities 

Alternative 

Expected 

Utility 

Base  Case 

0.451 

Alt  1 

0.413 

Alt  2 

0.435 

Alt  3 

0.466 

Now  alternative  three  ranks  first  followed  by  the  base  case.  One 
can  show  with  a  simple  computation  that  with  KC  valued  at  0.172, 
the  base  case  and  alternative  three  tie  for  the  first  rank  with 
equal  expected  utilities. 

7.  Summary.  With  reference  to  an  attached  compendium,  this  paper 
delineates  and  summarizes  practical  multiattribute  integration 
techniques  for  use  in  COEA.  Further,  it  provides  a  demonstration 
of  two  practical  multiattribute  techniques  for  use  in  COEA. 
Responses  to  Essential  Elements  of  Analysis  (EEAs)  as  well  as 
conclusions  follow. 

a.  Essential  Elements  of  Analysis  (EEA) 

EEA  1.  What  are  the  principal  multiattribute  techniques  for 
integrating  attributes  in  the  choice  of  alternatives? 


16 


A  number  of  techniques  exist  which  may  be  used  to  integrate 
attributes  in  the  choice  of  alternatives.  "Principal  techniques," 
for  the  purposes  of  the  Army  decision  making  environment,  comprise 
those  which  are  (1)  prescriptive  in  nature,  and  (2)  are  designed 
for  a  single  decision  maker  or  are  easily  adapted  to  group  use. 
Nineteen  such  techniques  are  set  out  in  the  compendium  of  this 
report . 

EEA  2.  What  are  the  most  practical  multiattribute  techniques 
for  use  in  COEA? 

Five  techniques  stand  out  as  probable  candidates  for  general 
use  in  COEA.  These  include  (1)  dominance,  (2)  the  conjunctive 
technique,  (3)  the  Analytic  Hierarchy  Process  (AHP) ,  (4) 
Multiattribute  Value  Theory  (MAVT) ,  and  (5)  Multiattribute  Utility 
Theory  (MAUT) .  Each  of  these  techniques  rates  highly  on  at  least 
four  of  the  six  criteria  set  out  earlier  in  this  report.  Dominance 
and  the  conjunctive  technique,  however,  are  perhaps  not  strictly 
accountable  to  the  criteria  of  "availability  of  application 
software"  or  "prevalence"  since  application  software  is  generally 
unnecessary,  and  since  such  techniques  may  be  applied  without 
explicit  acknowledgement  in  final  reports  or  journal  articles.  The 
other  three  techniques,  in  some  instances,  rate  poorly  with  respect 
to  "ease  of  use"  and/or  "data  requirements,"  but  they  should  not  be 
rejected  out  of  hand  in  such  instances  since  one  or  more  other 
criteria  (e.g.,  "availability  of  application  software")  may 
mitigate  a  poor  rating  on  "ease  of  use"  or  "data  requirements." 

Per  the  taxonomy  set  out  in  Appendix  B,  the  conjunctive  and 
dominance  techniques  are  relatively  easy  to  implement,  requiring  no 
information  on  the  relative  importance  of  attributes.  AHP,  MAVT, 
and  MAUT,  on  the  other  hand,  require  more  information  on  the 
relative  importance  of  attributes  and  are  computationally  more 
involved.  They  are  nonetheless  readily  applicable  to  the  typical 
COEA  problem,  and  may  be  described  as  "practical."  The  assessment 
level  of  the  techniques  ranges  from  ordinal  (dominance)  to 
stochastic  (MAUT) . 

EEA  3.  How  do  the  most  practical  techniques  compare  in  a 
typical  COEA  application? 

In  connection  with  the  Tow  Sight  Improvement  Program  (TSIP) 
COEA,  we  make  the  assumption  that  dominance  or  the  conjunctive 
technique  may  well  have  been  applied  early  in  the  process  of 
delimiting  alternatives.  Thus,  these  techniques  may  be  viewed  as 
"preparatory"  tools  to  be  used  early  in  the  analytic  process 
whereas  AHP,  MAVT,  or  MAUT  might  well  be  viewed  as  "finishing" 
machinery  in  that  process. 

AHP  and  MAUT  yield  similar  demonstration  results;  that  is, 
each,  under  "neutral"  circumstances,  highlights  the  Base  Case  as 
the  apparent  choice  for  a  decision  maker.  Each  technique  also 
shows  Alternative  3,  under  similar  circumstances,  to  be  a  "close 
second."  Moreover,  each  technique  allows  sensitivity  analysis  such 
that  an  analyst  may  demonstrate  to  a  decision  maker  the  impact  of 
differing  attribute  emphases,  or  values,  on  the  outcomes.  The 
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decision  maker  may  become  heavily  involved  in  the  analytic  process, 
for  either  MAUT  or  AHP,  as  the  analyst  requires  serious 
contemplation  of  attributes. 

AHP  and  MAUT  differ  in  several  respects.  Per  the  taxonomy  in 
the  Compendium,  AHP  involves  a  lower  level  of  assessment  as  well  as 
a  somewhat  less  burdensome  requirement  in  connection  with  attribute 
information.  Conceptually,  this  probably  makes  it  somewhat  more 
accessible  than  MAUT,  if  not  more  readily  carried  out.  MAUT  holds 
more  appeal  for  those  analysts  who  prefer  a  less  "plodding,"  more 
"elegant"  solution.  Both  techniques,  however,  now  enjoy  the 
advantage  of  generally  available  software  to  ease  computational 
pain  on  the  part  of  analysts.  Practically  speaking,  the  level  of 
information  available  would  reasonably  drive  the  choice  of  one 
technique  or  another  in  a  given  COEA. 

b.  Conclusions. 

(1)  A  number  of  multiattribute  integration  techniques, 
occupying  a  range  of  niches  in  a  taxonomy,  show  promise  for  use  in 
COEA.  Five  of  these  (refer  to  EEA  2)  may  currently  be  described  as 
"practical."  Others  may  prove  useful  in  light  of  future  research. 

(2)  Several  techniques  (e.g.,  AHP,  MAUT)  permit 
integration  of  attribute  estimates  or  outcomes  into  an  overall 
scheme  to  guide  a  decision  maker  in  the  choice  of  alternatives. 

(3)  Techniques  such  as  those  demonstrated  may  be  applied  - 
with  the  help  of  generally  available  software  -  to  COEA.  Such 
application  will  help  minimize  present  objectionable  analytical 
peculiarities  and  variances. 
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Appendix  B 

Compendium  of  Techniques 

1.  Introduction.  This  annotated  comoendium  of  techniques 
was  assembled  in  accord  with  the  scope  and  limitations  of 
the  study.  It  includes  (1)  a  brief  discussion  of  categories 
of  assessment  and  types  of  information  on  attributes,  (2)  a 
table  summary  of  techniques,  and  (3)  an  explication  of  those 
techniques.  Primary  references  are  Chankong  and  Haimes 
(1983),  Hwang  and  Yoon  (1981),  Keeney  and  Raiffa  (1976),  and 
MacCrimmon  (1973). 

2.  Assessment  Categories.  The  COEA  alternatives  are 
assessed  with  regard  to  each  attribute.  Assessments  may  use 
combat  simulation  models,  field  tests,  historical  databases, 
expert  opinion,  or  other  analytic  methods.  For  present 
purposes,  the  study  team  assumes  that  attributes  are 
monotonic  in  the  sense  that  either  more  is  always  better,  or 
less  is  always  better.  For  the  purpose  of  this  compendium 
the  type  of  assessment  is  categorized  as  ordinal (rank) , 
cardinal (numeric) ,  or  stochastic. 

a.  Ordinal  Assessment.  In  this  case  only  a  ranking  of 
alternatives  is  provided.  For  example,  suppose  three 
alternatives  A,  B,  and  C  are  to  be  compared  with  regard  to 
the  attribute,  total  comparative  life  cycle  cost.  An 
ordinal  assessment  may  yield  only  that  the  cost  of 
alternative  B  is  greater  than  the  cost  of  alternative  C  is 
greater  than  the  cost  of  alternative  A. 

b.  Cardinal (numeric) .  In  this  case  numeric  outcomes 
are  provided.  Comparing  the  three  alternatives  with  regard 
to  total  comparative  life  cycle  cost,  a  cardinal  assessment 
may  yield  $490  million  for  the  cost  of  alternative  A,  $850 
million  for  the  cost  of  alternative  B,  and  $640  million  for 
the  cost  of  alternative  C. 

c.  Stochastic.  In  this  case  a  probability  distribution 
is  provided  over  the  attributes  for  each  alternative. 
Continuing  the  example,  a  stochastic  assessment  may  yield 
that  the  cost  of  alternative  A  is  uniformly  distributed  from 
$440  million  to  $540  million,  that  the  cost  of  alternative  B 
is  uniformly  distributed  from  $800  million  to  $900  million, 
and  that  the  cost  of  alternative  C  is  uniformly  distributed 
from  $590  million  to  $690  million. 

d.  The  progression  of  assessment  types:  ordinal, 
cardinal,  stochastic,  constitutes  a  progression  of  levels  of 
measurement  and  generally  a  more  complex  assessment  process,. 

3.  Information  Types.  In  general  the  techniques  of  the 
compendium  require  information  on  relative  importance 
attached  to  the  attributes  by  the  decision  maker.  For  the 


purpose  of  this  compendium  the  type  of  information  on 
relative  importance  is  categorized  as  none,  ordinal,  simple 
weighting,  and  tradeoff. 

a.  No  information.  The  decision  maker  is  required  to 
provide  no  information  on  the  relative  importance  of  the 
attributes.  Techniques  for  this  type  are  applicable 
regardless  of  the  relative  importance  attached  to  the 
attributes  by  the  decision  maker. 

b.  Ordinal  information.  The  decision  maker  is  required 
to  rank  order  the  attributes  in  importance.  For  example, 
suppose  a  decision  problem  has  the  following  attributes: 

X,  total  comparative  life  cycle  cost,  Y,  loss  exchange  ratio 
in  defense,  and  Z,  loss  exchange  ratio  in  the  offense. 
Ordinal  information  from  the  decision  maker  may  yield  that  Z 
is  more  important  than  Y  is  more  important  than  X. 

c.  Simple  weighting  information.  The  decision  maker  is 
required  to  provide  information  from  which  numerical  weights 
for  each  attribute  can  be  derived.  These  weights  capture 
the  relative  importance  of  the  attributes.  Continuing  the 
example,  the  decision  maker  may  provide  information  which 
leads  to  the  weights:  0.23  for  total  comparative  life  cycle 
cost,  0.47  for  loss  exchange  ratio  in  the  defense,  and  0.72 
for  loss  exchange  ratio  in  the  offense.  In  order  for  the 
decision  maker  to  provide  meaningful  weights,  the 
attributional  scales  must  have  some  degree  of  comparability. 
If  the  original  assessment  scales  do  not  possess  this 
comparability,  new  scales  may  be  developed  and  corresponding 
outcomes  assessed  by  the  decision  maker. 

d.  Tradeoff  information.  The  decision  maker  is 
required  to  provide  enough  information  to  develop  a  value  or 
utility  function  over  the  complete  set  of  attributes.  The 
decision  maker  most  often  provides  this  information  in  the 
form  of  tradeoffs.  For  example,  given  a  cost  of  $450 
million,  a  loss  exchange  ratio  in  defense  of  2.0,  and  a  loss 
exchange  ratio  in  offense  of  0.5;  what  increase  in  cost 
would  be  traded  off  for  an  increase  to  1.0  for  the  loss 
exchange  ratio  in  the  offense? 

e.  The  progression  of  types  of  information:  none, 
ordinal,  simple  weighting,  tradeoff,  constitutes  a 
progression  of  levels  of  measurement  which  becomes 
increasingly  burdensome  to  the  decision  maker. 

4.  Using  the  three  types  of  assessment  and  the  four  types 
of  information  on  the  relative  importance  of  attributes,  we 
suggest  a  taxonomy  to  categorize  the  techniques  of  this 
compendium.  Table  1  illustrates  the  techniques  of  the 
compendium  in  the  context  of  this  taxonomy. 


Table  1 


Multiattribute  Techniques 


INFORMATION 
ON  RELATIVE 
IMPORTANCE 
OF 


ASSESSMENT 


ATTRIBUTES  | 

|  Ordinal  |  Cardinal  |  Stochastic 

None 

Dominance 

Maximin 

Maximax 

Majority  Rule 

Koler's 

Conjunctive 

Disjunctive 

Cost-Effect 
ratio (value) 

Stochastic 

dominance 

Cost-Effect 
ratio (utility) 

Ordinal 

Lexicographic 

Lexicographic 
with  minima 

Key  attribute 

Simple 

Weighting 

ELECTRE 

Permutation 

AHP 

SAW 

TOPSIS 

Tradeoff 

Value  Theory 
(MAVT) 

Utility  Theory 
(MAUT) 

Explications  of  the  techniques  of  the  table  follow. 


a.  Dominance.  Alternative  A  dominates  alternative  B  if 
it  is  better  on  at  least  one  attribute  and  at  least  as  good 
on  all  others.  Consider  the  following  two  attribute 
decision.  A1 ternat ive  A  has  cost  $500  million  and  loss 
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exchange  ratio  1.7.  Alternative  B  has  cost  $350  million  and 
loss  exchange  ratio  also  1.7.  Alternative  A  then  dominates 
alternative  B.  This  is  a  practical  technique  because  the 
attributes  are  assumed  to  be  monotonic (more  is  always 
better,  or  less  is  always  better) .  Of  all  the  techniques 
this  is  the  simplest  and  most  obvious. 

b.  Maximin.  In  this  technique  each  alternative  is 
represented  by  its  worst (minimum)  outcome  on  any  of  the 
attributes.  The  technique  then  select  the  alternative  with 
the  best (maximum)  outcome.  A  high  degree  of  comparability 
must  exist  among  the  attributional  scales.  If  this  is  not 
the  case  new  scales  may  be  developed.  The  decision  maker 
may  then  reassess  the  alternatives  using  these  new  scales. 

c.  Maximax.  In  this  technique  each  alternative  is 
represented  by  its  best (maximum)  outcome  on  any  of  the 
attributes.  The  technique  then  selects  the  alternative  with 
the  best (maximum)  outcome.  As  with  the  previous  technique  a 
high  degree  of  comparability  must  exist  among  the 
attributional  scales.  As  before  if  this  is  not  the  case  new 
scales  may  be  developed,  and  the  decision  maker  may  then 
reassess  the  alternatives  using  these  new  scales. 

d.  Majority  rule.  Alternative  A  ranks  before 
alternative  B  if  assessments  on  a  majority  of  the  attributes 
rank  alternative  A  before  alternative  B.  This  method  is 
applicable  if  the  decision  maker  ranks  the  attributes  equal 
in  relative  importance.  Although  conceptually  very  simple, 
this  technique  can  lead  to  an  overall  ranking  which  is  not 
transitive.  Consider  the  following  decision  situation: 

Rankings  of  Alternatives 

Attributes 

X  A  >  B  >  C 

Y  B  >  C  >  A 

Z  A  >  C  >  B 

Majority  rule  yields  an  overall  rank  of  A  >  B  >  C.  If  the 
ranking  on  attribute  Z  were  changed  to  C  >  A  >  B,  the 
overall  ranking  would  be  A  >  B,  B  >  C,  but  C  >  A.  Although 
people  certainly  exhibit  decision  behavior  which  is  not 
always  transitive,  for  prescriptive  purposes  techniques 
which  may  lead  to  intransitive  results  are  not  recommended. 

e.  Koler's  technique.  Arrow  &  Raynaud ( 1986) 
investigated  ranking  techniques  that  generalize  the  majority 
rule.  Their  concern  was  timely  business  decisions  that  have 
perhaps  hundreds  of  alternatives  and  hundreds  of  attributes. 
Ranking  alternatives  in  this  situation  is  termed  the 
industrial  outranking  problem.  The  technique  is  illustrated 


in  the  following  situation.  There  are  four  alternatives 
numbered  1  through  4,  which  are  compared  on  the  basis  of 
seven  attributes  denoted  A  through  G. 

Rankings  of  Alternatives 

Attributes 

A  3  >  2  >  4  >  1 

B  2  >  1  >  4  >  3 

C  1  >  3  >  2  >  4 

D  4  >  2  >  1  >  3 

E  4  >  3  >  1  >  2 

F  4  >  1  >  3  >  2 

G  1  >  2  >  3  >  4 

The  technique  is  based  on  the  outranking  matrix.  In  the  ith 
row  and  jth  column  of  this  matrix  is  the  number  of 
attributes  that  rank  alternative  i  before  alternative  j . 

The  technique  assumes  that  the  intensity  of  preference 
between  alternatives  i  and  j  is  a  strictly  increasing 
function  of  the  number  of  attributes  that  rank  i  before  j . 
The  outranking  matrix  for  the  above  data  is 

*  4  5  3 

3*34 
2  4  *  3 

4  3  4  * 

The  technique  proceeds  stepwise  to  identify  what  are  defined 
to  be  the  prudent  rankings.  These  are  transitive  rankings 
with  pairwise  comparisons  holding  for  the  largest  possible 
number  of  attributes.  First,  the  minimum  entry  of  each  row 
is  identified.  Next,  the  maximum  of  these  minima  is  chosen 
with  ties  broken  arbitrarily.  The  row  index  of  the  maximum 
indexes  the  alternative  ranked  at  this  step.  The  row  and 
column  with  this  index  are  deleted  from  the  matrix,  and  the 
process  is  repeated  until  the  matrix  is  empty.  The  above 
matrix  yields  the  rankings:  1243,  1324,  1432,  2413,  and 
4132. 

As  the  example  illustrates  the  technique  may  not  produce  a 
unique  ranking.  Although  the  technique  does  not  identify 
all  prudent  rankings,  if  there  is  a  unique  prudent  ranking 
the  technique  will  produce  it.  Also  if  the  majority  rule 
technique  would  yield  a  transitive  ranking,  Holer's 
technique  will  produce  it  as  the  unique  prudent  ranking. 
Because  of  its  simplicity  and  ease  of  use,  Holer's  technique 
has  potential  in  COEA  application.  However,  further 
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research  is  required  for  a  full  understanding  for  its 
application  in  a  COEA  environment. 

f.  Conjunctive  technique.  This  technique  requires  that 
minimum  acceptable  levels  be  established  for  each  attribute. 
Alternatives  are  then  categorized  as  acceptable  or 
unacceptable  according  to  whether  they  meet  or  better  all 
the  minima  or  not.  Like  dominance  this  is  a  simple  and 
obvious  technique.  For  use  with  COEA  establishing  the 
minimum  levels  may  present  difficulties.  Minima  for  some 
attributes  may  be  available  from  an  Operational  Requirements 
Document (ORD) . 

g.  Disjunctive  technique.  This  technique  requires  that 
exceptional  levels  be  established  for  each  attribute. 
Alternatives  are  then  categorized  as  acceptable  or 
unacceptable  according  to  whether  or  not  they  meet  or  better 
at  least  one  of  the  exceptional  levels.  As  with  the 
conjunctive  technique  determination  of  the  levels  limits  the 
practical  application  of  this  technique.  Exceptional  levels 
may  not  be  readily  established  in  most  COEA  applications. 

h.  Stochastic  dominance.  This  technique  is  the 
stochastic  analogue  of  the  previous  dominance  technique. 
Assume  assessments  have  resulted  in  probability 
distributions,  one  for  each  alternative,  over  the 
attributes.  Alternative  A  stochastically  dominates 
alternative  B,  if  the  outcomes  of  A  are  better  than  those  of 
B  with  probability  one.  When  the  individual  attributes  have 
independent  probability  distributions  this  technique  is 
easily  applied. 

i.  Lexicographic  technique.  This  is  the  ordering 
technique  of  a  dictionary.  Alternatives  are  first  ranked 
according  to  the  most  important  attribute.  If  two 
alternatives  tie  on  this  attribute,  they  are  then  ranked 
according  to  the  second  most  important  attribute.  If  they 
tie  according  to  this  attribute  the  third  most  important 
attribute  is  used,  and  so  on.  Consider  the  decision 
situation  with  alternatives  A,  B,  and  C  and  attributes  X,  Y, 
and  Z  where  the  order  of  the  attributes  in  relative 
importance  is  X  >  Y  >  Z. 


Rankings  of  Alternatives 

Attributes 

X  A  >  B  =  C 

Y  B  =  A  >  C 

Z  C  >  B  >  A 

Lexicographic  technique  yields  an  overall  ranking  of 
A  ■  B  ->  C. 


In  COEA  the  lexicographic  technique  eight  be  sore  useful  if 
a  practical  level  of  difference  vere  established  for  each 
attribute.  Two  alternatives  would  be  ranked  equal  on  an 
attribute  if  their  outcomes  vere  within  the  corresponding 
practical  level  of  difference.  Practical  levels  of 
difference  would  take  into  account  the  accuracy  of  the 
corresponding  assessment  technique.  For  example,  a 
practical  level  of  difference  for  a  cost  attribute  might  be 
determined  to  be  $10  million.  Alternative  A  with  a  cost  of 
$523  million  and  alternative  B  with  a  cost  of  $517  million 
would  then  be  ranked  equal  on  the  cost  attribute,  for 
application  of  this  technique. 

j.  Lexicographic  with  minima.  As  in  the  conjunctive 
technique  minimum  acceptable  levels  must  be  established  for 
each  attribute.  Alternatives  are  first  ranked  according  to 
the  most  important  attribute,  if  its  minimum  level  is  not 
met.  If  the  minimum  level  for  the  most  important  attribute 
is  met  but  the  minimum  level  for  the  second  most  important 
attribute  is  not,  the  alternatives  are  ranked  according  to 
the  second  most  important  attribute.  If  the  minimum  level 
of  the  second  most  important  attribute  is  met  but  the 
minimum  level  of  the  third  most  important  attribute  is  not 
met,  alternatives  are  ranked  by  the  third  most  important 
attribute,  and  so  on.  This  technique  is  similar  to  the  more 
complex  Elimination  by  Aspects  (EBA) . 

k.  Key  attribute.  Minimum  acceptable  levels  are 
established  for  all  attributes,  except  one,  the  key 
attribute.  Alternatives  which  do  not  meet  or  better  all 
minima  are  unacceptable.  Acceptable  alternatives  are  then 
ranked  according  to  the  key  attribute.  In  use  with  COEA 
minimum  levels  might  be  established  for  all  attributes 
except  cost,  with  cost  identified  as  the  key  attribute. 
Alternatives  which  do  not  meet  the  minimum  levels  are 
eliminated  as  unacceptable.  The  acceptable  alternatives  are 
then  ranked  by  cost. 

l.  Permutation.  This  technique  successively  computes 
an  index  value  for  each  possible  ranking (permutation)  of 
alternatives.  The  ranking  with  the  highest  index  is 
recommended.  Consider  the  decision  situation  with 
alternatives  A  and  B  and  attributes  X  and  Y. 

Rankings  of  Alternatives 

Attributes  and  Weight 

X  0.4  A  >  B 

Y  0.6  A  <  B 

Consider  the  overall  ranking(permutation)  of  A  >  B.  Since 
A  >  B  on  attribute  X,  attribute  X  constitutes  the  set  of 
concordance.  Since  A  <  B  on  attribute  Y,  attribute  Y 
constitutes  the  set  of  discordance.  The  index  for  the 
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overall  ranking  is  the  weight  of  attribute  X (concordance) 
minus  the  weight  of  attribute  Y (discordance) ,  -0.2. 

Likewise,  consider  the  overall  ranking (permutation)  of 
A  <  B.  Since  A  >  B  on  attribute  X,  attribute  X  constitutes 
the  set  of  discordance.  Since  A  <  B  on  attribute  Y, 
attribute  Y  constitutes  the  set  of  concordance.  The  index 
for  the  overall  ranking  is  the  weight  of  attribute  Y 
(concordance)  minus  the  weight  of  attribute  X (discordance) , 
0.2.  Since  the  overall  ranking (permutation)  A  <  B  has  the 
highest  index  it  is  recommended. 

m.  ELECTRE.  This  technique  (Elimination  et  Choix 
Translation  Realite)  is  similar  to  the  permutation  technique 
in  that  indices  involving  concordances  and  discordances  are 
computed.  However,  indices  are  only  computed  for  pairs  of 
alternatives.  Alternatives  are  then  ordered  based  on  these 
indices.  Unlike  the  permutation  method  ELECTRE  requires  the 
decision  maker  to  set  arbitrary  threshold  values  for  the 
indices.  The  resulting  ranking  is  in  general  neither 
complete  nor  transitive.  The  computations  required  in  the 
permutation  technique  or  ELECTRE  could  easily  be  implemented 
in  commercially  available  spreadsheets. 

n.  AHP.  The  Analytic  Hierarchy  Process (AHP)  is 
described  in  Saaty(1980) .  Its  axiomatic  foundations  are 
given  in  Saaty(1989).  The  two  key  features  of  AHP  are  a 
hierarchy  of  attributes  and  the  eigenvector  weighting 
method.  The  weighting  method  typically  uses  as  scale  values 
the  integers  1  to  9  and  their  reciprocals.  Consider  the 
decision  situation  with  alternatives  A  and  B  and  a  two  level 
hierarchy  of  attributes.  Attribute  X  is  cost,  and  attribute 
Y  is  effectiveness.  Subattribute  XI  is  peace  time  cost  and 
subattribute  X2  is  war  time  cost.  Subattributes  Yl,  Y2,  and 
Y3  are  effectiveness  under  scenarios  one,  two,  and  three, 
respectively. 


X 


XI  X2 


A  B  A  B 


Y 


Yl  Y2  Y3 


A  B  A  B  A  B 


This  hierarchy  requires  eight  distinct  scalings. 

Alternatives  are  pairwise  compared  for  their  contribution  to 
each  of  the  five  lowest  level  attributes.  Next,  the 
subattributes  XI  and  X2  are  compared  for  the  importance  of 
their  contribution  to  the  attribute  X,  and  the  attributes 
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Yl,  Y2,  and  Y3  are  compared  for  their  contribution  to  the 
attribute  Y.  Finally,  the  attributes  X  and  Y  are  compared 
for  their  importance  to  the  overall  decision. 

Table  2  illustrates  the  scale  used  to  assess  the  pairwise 
importance  of  the  attributes  or  alternatives  in  the 
hierarchy. 


Table  2 
AHP  Scale  of 
Pairwise  Importance 

Importance  of  one 

factor  over  another  Value 


Equal 
Weak 
Strong 
Very  Strong 
Absolute 


1 

2 

3 

4 

5 

6 

7 

8 
9 


The  AHP  importance  scale  also  makes  use  of  the  reciprocals 
of  the  values  in  Table  2.  For  example,  if  the  pairwise 
importance  of  factor  1  over  factor  2  is  judged  to  be  weak 
and  valued  at  3,  the  pairwise  importance  of  factor  2  over 
factor  1  is  valued  at  1/3. 

Pairwise  importances  for  a  set  of  attributes  or  alternative 
are  assessed  in  a  matrix  format.  For  example,  consider  the 
relative  assessment  of  the  three  attributes:  Yl,  Y2,  Y3. 
Suppose  that  Yl  is  judged  to  be  strongly  more  important  than 
Y2  and  absolute  more  important  than  Y3.  Suppose  also  that 
Y2  is  judged  to  be  weakly  more  important  than  Y3.  Table  3 
illustrates  these  scale  values  in  matrix  format. 

Table  3 

Pairwise  Comparisons  of 
Attributes  Y1,Y2,Y3 


Yl 

Y2 

Y3 

Yl 

1 

5 

9 

Y2 

1/5 

1 

3 

Y3 

1/9 

1/3 

1 

U-9 


The  principal  eigenvalue  or  largest  real  eigenvalue  of  this 
matrix  is  3.029.  Its  associated  eigenvector,  normalized  so 
its  components  sum  to  one,  is  (0.7514,0.1782,0.0704).  The 
eigenvector  weighting  method  assigns  Y1  a  weight  of  0.7514, 
Y2  a  weight  of  0.1782,  and  Y3  a  weight  of  0.0704.  Note  that 
the  assessed  values  of  Table  3  are  not  completely 
consistent.  If  the  5  entry  and  1/5  entry  were  changed  to  3 
and  1/3,  respectively,  the  scaling  becomes  completely 
consistent  with  a  principal  eigenvalue  of  3.  In  general  the 
principal  eigenvalue  is  at  least  as  large  as  the  dimension 
of  the  matrix,  in  this  example,  3.  The  greater  the 
eigenvalue  the  more  inconsistent  the  datal 

Once  all  weights  are  computed  by  the  eigenvector  method  an 
overall  weight  for  each  alternative  is  computed.  Weights 
for  each  alternative  are  computed  linearly  up  the  hierarchy. 
Continuing  the  example  suppose  the  assessments  and 
eigenvector  method  lead  to  the  following  weights,  symbolized 
with  lower  case  letters. 


X  (w)  Y  (v) 


XI  (wl)  X2  (w2 )  Y1  (Vl)  Y2  (v2 )  Y3  (v3 


AB  ABABABAB 

wal  wbl  wa2  wb2  val  vbl  va2  vb2  va3  vb3 

The  overall  weight  of  alternative  A  is 

w*(wl*wal  +  w2*wa2)  +  v*(vl*wal  +  v2*va2  +  v3*va3) 

The  overall  weight  of  alternative  B  is 

w*(wl*wbl  +  w2*wb2)  +  v*(vl*wbl  +  v2*vb2  +  v3*vb3) 

Alternatives  are  now  ranked  by  overall  weight.  Although  the 
decision  maker  is  required  to  make  a  number  of  assessments, 
the  nine  point  scale  simplifies  the  required  scaling. 
Commercial  personal  computer  software  is  available  to 
implement  AHP.  The  only  difficult  computations  are  those  to 
determine  the  eigenvalues  and  eigenvectors.  Most  numerical 
analysis  software  libraries  have  routines  to  perform  these 
computations.  Eigenvalue  and  eigenvector  computations  for 
this  study  were  accomplished  using  IMSL,  a  commercially 
available  FORTRAN  library.  Computation  of  overall  weights 
for  alternatives  could  easily  be  implemented  in  commercially 
available  spreadsheets. 
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o.  Value  Theory.  Multiattribute  value  theory (MAVT) 
encompasses  a  number  of  techniques  that  have  a  common 
theoretical  basis.  This  basis  is  measurement  theoretic  and 
relies  on  the  concept  of  a  preference  or  value  function.  A 
preference  function  is  intended  to  measure  a  decision 
maker's  preference  for  various  combinations  of  outcomes  on 
the  attributes.  Consider  an  example  with  alternatives,  A 
and  B,  and  attributes,  XI,  X2,  X3,...,Xn.  Alternative  A  has 
outcomes  xla,  x2a,  x3a,..,xna,  and  alternative  B  has 
outcomes  xlb,  x2b,  x3b,...xnb.  If  the  decision  maker's 
preference  over  the  attributes  is  represented  by  a 
preference  function  F,  alternative  A  is  ranked  before 
alternative  B  if 

F(xla,x2a,x3a, . .  .xna)  >  Ffx2,b,x2b,x3b, . .  .xnb) 

Typically  value  is  only  an  ordinal  concept  hence  the 
function  F  is  only  unique  up  to  a  strictly  monotonically 
increasing  transformation.  See  Roberts (1979)  for  the 
theory.  Commercial  personal  computer  software  is  available 
to  implement  multiattribute  value  theory. 

(1)  Indifference.  The  concept  of  indifference  provides 
a  technique  for  investigating  preference  functions.  An 
indifference  set  consists  of  all  combinations  of  outcomes 
which  are  equally  valued  by  the  preference  function.  In  two 
dimensions  the  indifference  set  is  simply  a  curve.  For 
example,  cos*  and  effectiveness  outcomes  (Ca,Ea)  and  (Cb,Eb) 
are  on  the  same  indifference  curve  of  the  preference 
function  F  if 


F(Ca,  Ea)  -  F(Cb,  Eb) 

This  means  that  the  decision  maker  is  indifferent  with 
regard  to  alternative  A  (with  cost  Ca  and  effectiveness  Ea) 
and  alternative  B  (with  cost  Cb  and  effectiveness  Eb) . 
Algebraically  an  indifference  curve  is  defined  by  the 
relationship: 


F (Cost,  Effectiveness)  *  constant 

Graphically  an  indifference  curve  may  be  represented  in  the 
cost-effectiveness  plane  as  shown  in  Figure  1  below. 
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Effectiveness 


(Cb.Eb) 


(Ca,Ea) 


-i - » 

Cost 

Figure  1.  An  Indifference  Cur«/e 

All  combinations  of  cost  and  effectiveness  along  the  curve 
of  Figure  1  are  equally  preferred. 

(2)  Some  properties  of  preference  functions.  In  general 
preference  relationships  will  vary  by  individual  decision 
makers.  Consequently  there  is  no  one  correct  preference 
function  for  ranking  alternatives.  However,  there  may  be 
reasonable  behavioral  rationale  for  limiting  the  types  of 
preference  functions  and  thus  constraining  the  shapes  of  the 
indifference  sets.  Some  propc.  :*ies  of  preference  functions 
are  illustrated  assuming  only  two  attributes,  cost  and 
effectiveness . 

It  is  likely  that  a  decision  maker  prefers  alternative  A  to 
alternative  B  if  alternative  A  is  at  least  as  effective  but 
less  cost aV  or  more  effective  but  no  more  costly.  If  this 
is  the  case  the  decision  maker's  preference  relationship 
exhibits  the  property  of  dominance.  This  property  is  simply 
the  dominance  technique  previously  discussed  now  in  the 
context  of  a  preference  function.  Dominance  requires  that 
the  preference  function  be  increasing  with  increasing 
effectiveness  or  decreasing  cost. 

The  following  two  typical  preference  properties (PP)  are 
discussed  in  detail  in  Keeney  and  Raiffa(1976) 

PP1:  At  a  fixed  level  of  cost  as  effectiveness 

increases  the  decision  maker  will  pay  l^ss  for  a 
fixed  increase  in  effectiveness 


and 


PP2 :  At  a  fixed  level  of  effectiveness  as  cost 
increases  the  decision  maker  will  pay  less  for  a 
fixed  increase  in  effectiveness 

These  tv/o  properties  are  consistent  with  the  concave  upward 
indifference  curves  illustrated  below: 


Effectiveness 


Figure  2.  Indifference  Curve  with  PP1  and  PP2 


The  indifference  curves  of 
those  of  Figure  3  below. 


Figure  2  may  be  contrasted  with 


Effectiveness 


Figure  3.  Indifference  Curve  with  PP3  and  PP4 


The  concave  downward  curves  of  Figure  3  are  found  when  the 
following  two  preference  properties  hold 

PP3 :  At  a  fixed  level  of  cost  as  effectiveness 
increases  the  decision  maker  will  pay  more  for  a 
fixed  increase  in  effectiveness 


PP4 :  At  a  fixed  level  of  effectiveness  as  cost 
increases  the  decision  maker  will  pay  more  for  a 
fixed  increase  in  effectiveness 


and 


For  more  on  shapes  of  indifference  curves  see  MacCrimmon  and 
Wehrung ( 1977) . 

A  simpler  type  of  indifference  curve  occurs  when  the 
decision  maker  will  pay  the  same  fixed  increase  in  cost  for 
the  same  fixed  increase  in  effectiveness  anywhere  in  the 
cost-effectiveness  plane.  Equivalently,  the  rate  of 
substitution  of  effectiveness  for  cost  is  constant.  In  this 
case  preference  can  be  represented  by  a  linear  function  of 
cost  and  effectiveness  and  the  indifference  curves  torn  a 
family  of  parallel  lines  as  illustrated  in  Figure  4.  The 
slope  of  the  parallel  lines  equals  the  constant  rate  of 
substitution  of  effectiveness  for  cost. 


Effectiveness 


Figure  4.  Linear  Indifference  Curves 

In  this  linear  case  alternatives  can  simply  be  compared  by 
the  ratio  of  their  difference  in  effectiveness  to  their 
difference  in  cost.  Let  R  be  the  constant  rate  of 
substitution  of  effectiveness  for  cost.  For  alternatives  A 
(Ca,Ea)  and  B  (Cb,Eb)  with  B  the  more  effective  let 

S  =  (Eb  -  Ea)  /  Cb  -  Ca) 

Then  A  is  preferred  if  S  <  R,  B  is  preferred  if  S  >  R,  and  A 
and  B  are  equally  preferred  if  S  =  R. 

(3)  Additive  Value.  In  general,  determination  of  a 
decision  maker's  preference  function  is  a  complex  task. 
Keeney  and  Raiffa(1976)  describe  the  details  of  the  process. 
Simplification  occurs  when  linearity  properties  hold.  In 
this  situation  the  decision  maker's  preferences  may  be 
represented  by  a  function  F  that  decomposes  as 

F(X1,X2,X3 _ _ Xn)  =  F1(X1)  +  F2(X2)  +  F3(X3)  +...+  Fn(Xn) 
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a  sum  of  the  single  attribute  preference  functions 
FI , F2 , F3 , . . . , Fn.  Determination  of  the  single  attribute 
functions  is  in  theory  a  simpler  task  than  determining  a 
multiattribute  function.  The  two  attribute  example  with  a 
constant  rate  of  substitution  of  cost  for  effectiveness  is  a 
special  case  of  such  a  linear  decomposition.  An  alternate 
form  has  the  function  decomposing  as 


F (XI , X2 , X3 , . . . , Xn)  = 


and 


wl*f 1 (XI) +w2*f2 (X2 ) +w3  *f 3 (X3)+. . .+wn*fn(Xn) 
wl+w2+w3+. . ,+wn  =  1 


A  number  of  techniques  simply  assume  a  linear  decomposition 
of  the  preference  function.  A  justification  for  this 
assumption  is  that  additive  value  yields  a  robust 
approximation.  See  Dawes  and  Corrigan ( 1974 ) . 


p.  SAW.  The  technique  of  Simple  Additive 
Weighting (SAW)  is  a  common  additive  value  technique.  This 
technique  assumes  a  linear  decomposition  of  the  preference 
function.  However,  rather  than  determining  the  single 
attribute  preference  functions  explicitly,  the  technique 
relies  on  comparable  preference  scales  for  the  attributes 
and  weights  for  their  relative  importance.  For  example, 
suppose  the  assessed  outcomes  for  alternative  A  are 
xl,x2,x3, . . . ,xn.  The  decision  maker  converts  these  to 
comparably  scaled  preference  outcomes  yl,y2,y3, . . . ,yn.  The 
decision  maker  also  provides  relative  weights; 

wl,w2,w3, . . . ,wn  for  the  attributes.  The  preference  function 
is  then  evaluated  for  alternative  A  by; 

F(A)  =  wl*yl  +  w2*y2  +  w3*y3  +...+  wn*yn 

Although  differing  in  theory,  the  form  here  is  very  similar 
to  that  of  AHP  .  The  application  may  be  even  more  similar 
if  the  eigenvector  weighting  process  is  used  to  determine 
the  scaled  outcomes  yi's  and  the  attribute  weights  wi's. 

q.  Cost-effectiveness  ratios.  If  there  is  only  a  cost 
attribute  and  an  effectiveness  attribute,  alternatives  could 
be  ranked  by  their  cost-effectiveness  ratios.  From  the 
standpoint  of  value  theory,  ranking  by  cost-effectiveness 
ratios  assumes  a  ratio  preference  function  F  as  follows 

F(Cost,  Effectiveness)  =  Effectiveness  /  Cost 

Alternative  A,  with  cost  Ca  and  effectiveness  Ea  is  ranked 
before  alternative  B,  with  cost  Cb  and  effectiveness  Eb  if 


F(Ca ,  Ea)  =  Ea /Ca  >  F(Cb,  Eb)  =  Eb/Cb 


Preference  that  is  represented  by  the  ratio  preference 
function  also  has  an  additive  representation.  Since  the 
logarithmic  transformation  is  strictly  monotonically 
increasing  the  functions 


Effectiveness  /  Cost 


and 

log (Effectiveness  /  Cost) 

yield  the  same  preference  ordering.  However  the  logarithmic 
form  decomposes  as 

log (Effectiveness  /  Cost)  =  log (Effectiveness) 


+  log(l  /  Cost) 

The  cost-effectiveness  ratio  is  an'  example  of  a  derived 
measure,  wherein  fundamental  measures,  cost  and 
effectiveness,  are  combined  mathematically  to  produce  a  new 
or  derived  measure.  A  standard  example  of  a  derived  measure 
is  density,  the  ratio  of  the  fundamental  measures  mass  and 
volume.  The  same  fundamental  measures  can  be  combined  in 
different  ways  to  produce  distinct  derived  measures.  For 
example,  the  length,  width,  and  height  of  rectangular  boxes 
may  be  combined  to  produce  either  the  surface  area  measure 
or  the  volume  measure.  Care  should  be  taken  in  assuming 
that  the  mathematics  of  any  particular  derivation 
consistently  reflects  a  decision  maker's  preference 
relationships  over  the  fundamental  measures.  Ranking  by 
cost-effectiveness  ratios  makes  this  assumption. 

Indifference  curves  of  the  ratio  preference  function  are 
rays  emanating  from  the  origin  as  shown  in  Figure  5. 


Effectiveness 
( 1000K) 


Figure  5.  Indifference  Curves  of  the  Ratio  Preference 

Function 
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As  shown  in  the  figure  cost  and  effectiveness  combinations 
(1.0  billion  dollars  and  2000  kills)  and  (2.0  billion 
dollars  and  4000  kills)  lie  on  the  same  indifference  curve. 
Equivalently,  the  decision  maker  is  indifferent  with  regard 
to  the  alternative  with  cost  1.0  billion  dollars  and 
effectiveness  2000  kills  and  the  alternative  with  cost  2.0 
billion  dollars  and  effectiveness  4000  kills.  Any  other 
combination  of  cost  and  effectiveness  with  the  same  kill  to 
dollar  ratio  would  lie  on  this  indifference  curve. 

Preference  property  one  (PP1)  holds  for  the  ratio  preference 
function.  However,  this  function  also  exhibits  preference 
property  four  (PP4) .  This  fact  is  illustrated  in  the  next 
figure. 


Figure  6.  Amounts  Paid  for  Fixed  Increase  in  Effectiveness 

In  Figure  6  alternatives  A  and  B  have  the  same 
effectiveness.  Alternative  B  has  the  greater  cost.  An 
increase  of  E  in  effectiveness  is  worth  an  increase  of  C 
in  cost  at  A  and  an  increase  of  D  in  cost  at  B.  It  can  be 
seen  from  the  figure  that  D  is  greater  than  C.  Thus  more 
is  being  paid  for  a  fixed  increase  in  effectiveness  at  a 
higher  level  of  cost. 

The  advantage  of  cost-effectiveness  ratios  is  that  the 
decision  maker  is  required  to  provide  little  or  no 
information.  The  disadvantage  is  that  the  decision  maker 
must  accept  the  unique  properties  which  this  technique 
requires. 

r.  TOPSIS.  The  Technique  for  Order  Preference  by 
Similarity  to  Ideal  Solution (TCPSIS)  develops  a  preference 
function  based  on  distances  between  alternatives  as 
represented  in  attribute  space.  This  technique  requires 
only  that  the  decision  maker  give  relative  weights  for  the 
attributes.  These  weights  are  normalized  and  then  combined 
with  normalized  outcomes  on  the  attributes  to  form  the 
decision  outcomes.  The  best  decision  outcomes  become  the 
outcomes  of  the  ideal  alternative.  The  worse  decision 
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outcomes  become  the  outcomes  of  the  negative  ideal 
alternative.  Alternatives  are  then  ranked  using  their 
distances  from  these  two  hypothetical  alternatives. 

Consider  an  example  with  alternatives  A,  B,  and  X  and  two 
attributes,  cost  and  effectiveness.  The  situation  is 
illustrated  in  Figure  7  below.  Alternatives  are  represented 
graphically  by  their  normalized  weighted  outcomes  (Ca,Ea), 
(Cb,Eb),  and  (Cx,Ex) .  The  ideal  alternative  I  has  the  best 
cost,  Cb,  and  the  best  effectiveness,  Ea.  The  negative 
ideal  alternative  N  has  the  worst  cost,  Ca,  and  the  worst 
effectiveness,  Eb. 


Effectiveness 


I (Cb, Ea) 


A (Ca , Ea) 


Cost 

Figure  7.  Computing  the  TOPSIS  Index 

For  alternative  X  the  distance,  ID,  between  it  and  the 
ideal  alternative  and  the  distance,  ND,  between  it  and  the 
negative  ideal  alternative  are  computed.  The  index  of 
alternative  X  is  defined  to  be 

I(X)  =  ND  /  (ID  +  ND) 

The  indices  of  the  other  alternatives  are  computed 
similarly.  Alternatives  are  then  ranked  according  to  their 
indices. 

TOPSIS  indifference  curves,  assuming  that  cost  and 
effectiveness  outcomes  have  been  normalized  and  weighted, 
are  illustrated  in  Figure  8  below.  One  indifference  curve 
is  the  perpendicular  bisector,  L,  of  the  line  segment 
joining  the  ideal  and  negative  ideal  alternatives.  Other 
indifference  curves  are  segments  of  circles (Circles  of 
Apollonius) .  Segments  above  L  are  concave  upward  and 
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segments  below  L  are  concave  downward.  Preference 
properties  PP1  and  PP2  hold  above  L.  Preference  properties 
PP3  and  PP4  hold  below  L.  This  situation  should  not  be 
assumed  to  hold  for  a  decision  maker's  preference  without  a 
more  detailed  examination. 


Effectiveness 
(normalized 
&  weighted) 


I  A 


Cost 

(normalized  6  weighted) 

Figure  8.  TOPSIS  Indifference  curves 

The  next  example  illustrates  that  TOPSIS  does  not  satisfy 
the  property  of  independence  of  irrelevant  alternatives (see 
Luce  and  Raiffa,  1957).  Assume  that  the  decision  maker 
equally  weights  cost  and  effectiveness.  Table  2  gives 
outcomes (suitably  scaled)  for  the  cost  and  effectiveness  of 
four  alternatives,  A  through  D.  It  also  gives  the  TOPSIS 
index. 


Table  2 

Initial  TOPSIS  Results 


Alternative 

Cost 

Effectiveness 

TOPSIS 

A 

4.0 

4.0 

0.624 

B 

4.1 

10.0 

0.874 

C 

6.0 

12.0 

0.860 

D 

16.0 

12.1 

0.376 

Alternative  B  is  ranked  first,  C  second,  A  third,  and  D 
last.  Suppose  new  data  or  further  analysis  result  in  a 
change  in  the  effectiveness  of  alternative  D  from  12.1  to 
13.1.  The  revised  data  together  with  the  new  indices  are 
given  in  the  Table  4. 
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Table  4 

Updated  TOPSIS  Results 


Alternative 

Cost 

Effectiveness 

TOPSIS  Index 

A 

4.0 

4.0 

0.603 

B 

4.1 

10.0 

0.828 

C 

6.0 

12.0 

0.846 

D 

16.0 

13.1 

0.397 

The  rank  of  alternative  D  is  not  changed,  it  is  still  a 
distant  fourth.  However,  alternative  C  is  now  ranked  first 
and  B  second,  reversing  the  previous  pairwise  ranking.  Thus 
the  ranking  of  two  alternatives  has  switched  because  of  a 
change  to  the  outcome  of  a  poorly  ranked,  or  irrelevant, 
alternative. 

An  advantage  of  TOPSIS  is  that  it  requires  little 
information  from  the  decision  maker.  Disadvantages  are  that 
it  requires  the  acceptance  of  a  unique  set  of  indiffA:ence 
curves  and  the  rejection  of  the  property  of  independence  of 
irrelevant  alternatives. 

The  next  example  illustrates  that  TOPSIS  and  cost- 
effectiveness  ratios  may  yield  different  results.  Table  5 
gives  cost  and  effectiveness  outcomes  for  three 
alternatives.  In  addition  the  TOPSIS  index  and 
effectiveness  to  cost  ratio  are  given. 

Table  5 

TOPSIS  and  Cost-effectiveness  Ratios 


Alternative 

Cost 

Effectiveness 

Eff/Cost 

TOPSIS 

Index 

A 

1.0 

4.0 

4.0 

0.719 

B 

2.0 

7.0 

3.5 

0.877 

C 

12.0 

8.0 

0.667 

0.281 

Using  the  effectiveness  to  cost  ratios,  alternative  A  ranks 
first  and  alternative  B  second.  However,  using  the  TOPSIS 
index,  alternative  B  ranks  first  and  alternative  A  second. 

s.  Utility  Theory.  Multiattribute  utility  theory  is  a 
stochastic  analogue  of  value  theory.  The  value  or 
preference  function  is  replaced  by  the  utility  function. 
Outcomes  of  the  alternatives  are  assessed  with  probability 
distributions  rather  than  with  point  estimates.  The  theory 
and  application  are  described  in  Keeney  and  Raif fa ( 1976) . 

This  technique  is  subsumed  by  the  method  of  decision 
analysis.  A  basic  axiom  of  decision  analysis  is 
maximization  of  expected  utility.  Consider  an  example  with 
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alternatives,  A  and  B,  and  attributes,  XI,  X2,  X3,...,Xn. 
Alternative  A  has  a  probability  distribution  of  outcomes 

fa(xl,  x2 ,  x3,..,xn) 

and  alternative  B  has  a  probability  distribution  of  outcomes 

fb(xl,  x2,  x3,...xn) 

If  the  decision  maker's  preference  over  the  attributes  is 
represented  by  utility  function  U,  the  expected  utility  of 
alternative  A  is 

J*U(xl,  x2 ,  x3,..,xn)  *  fa(xl,  x2,  x3,..,xn) 

and  the  expected  utility  of  alternative  B  is 

Ju(xl,  x2,  x3,..xn)  *  fb(xl,  x2,  x3,...xn) 

Alternative  are  then  ranked  by  their  expected  utility. 

One  of  the  strengths  of  MAUT  is  that  in  the  context  of 
decision  analysis  it  accommodates  a  coherent  integration 
with  a  theory  of  probability  and  statistical  inference. 

This  allows  for  a  rigorous  mathematical  treatment  of 
uncertainty  and  risk. 

(1)  Additive  Utility.  Determination  of  a  decision 
maker's  utility  function  is  a  highly  complex  task  as  Keeney 
and  Raiffa(1976)  show.  The  process  greatly  simplifies  if  a 
property  of  additive  independence  holds.  In  this  situation 
the  decision  maker's  utility  may  be  represented  by  a 
function  U  that  decomposes  as 

U (xl , x2 , x3 , . . . , xn)  = 

wl*ul(xl)  +  w2*u2(x2)  +  w3*u3 (x3)  +  ...+  wn*un(xn) 

a  sum  of  the  single  attribute  utility  functions  ul,  u2, 
u3,..,un.  Determination  of  the  single  attribute  functions 
is  in  theory  a  simpler  task  than  determining  a 
multiattribute  function. 

(2)  Multilinear  Utility.  A  somewhat  more  complex 
decomposition  results  when  a  property  known  as  utility 
independence  holds.  See  Theorem  6.3  of  Keeney  and  Raiffa. 

In  this  case  the  decision  maker's  utility  may  be  represented 
by  a  function  U  that  decomposes  as 

U(xl,x2,x3, . . . , xn)  =  Z.wi*ui(xi) 

+  Z  wij*ui (xi) *uj (xj) 

+  ... 

+  Z  wij.. .n*ui(xi)*uj (xj)*. . .*un(xn) 
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where  as  with  the  additive  decomposition  the  ui  are  single 
attribute  utility  functions.  Additive  utility,  of  course, 
is  simply  a  special  case  of  the  multilinear.  A  second 
special  case,  the  multiplicative,  is  presented  in  Theorem 
6.1  of  Keeney  and  Raiffa. 

(3)  Probabilistic  Independence.  Just  as  conditions  of 
independence  simplify  the  modeling  of  utility  they  also 
simplify  the  modeling  of  the  probability  distributions  of 
outcomes.  For  any  particular  alternative  the  attributes 
may  be  probabilistically  independent.  In  .this  case  the 
multivariate  probability  distribution  of  outcomes  will 
decompose  as  a  product 

f (xl,x2, . . ,xn)  =  f 1 (xl) *f 1 (x2) * . . . *fn (xn) 
of  univariate  probability  distributions,  fi. 

(4)  Risk  Attitude.  A  single  attribute  utility  function  u 
is  often  scrutinized  by  considering  lotteries  <x,y>  with 
outcomes  x  and  y  having  an  equal  chance  of  occurring.  The 
expected  value  of  such  a  lottery  is  simply 

(x  +  y)  /  2 

The  expected  utility  of  this  lottery  is 

(u(x)  +  u(y) )  /  2 

The  value  z  such  that  u(z)  is  equal  the  expected  utility  is 
called  the  certainty  equivalent  of  the  lottery. 

If  a  decision  maker's  utility  is  such  that  the  certainty 
equivalent  and  the  expected  value  are  equally  preferred,  the 
decision  maker  is  said  to  be  risk  neutral.  In  this  case  the 
utility  function  is  linear.  If  the  expected  value  is 
preferred  to  the  certainty  equivalent,  the  decision  maker  is 
risk  averse.  If  the  certainty  equivalent  is  preferred  to 
the  expected  value,  the  decision  maker  is  risk  prone. 

(5)  Of  all  the  techniques  discussed  utility  theory  is 
the  most  theoretically  sound;  however,  it  is  also  the  most 
complex.  The  greater  the  number  of  attributes  the  greater 
is  the  complexity.  Increasing  availability  of  application 
software  is  making  this  technique  more  practical.  The 
recently  developed  IDEA  software,  Whitfield  et  al. (1989) ,  is 
such  an  example. 

t.  Cost-effectiveness  ratios.  Previously  cost- 
ef  feet  iveness  ratios  were  viewed  from  the  stand  point  of 
value  theory.  They  can  also  be  viewed  from  the  stand  point 
of  utility  theory.  In  this  case  it  is  assumed  that  a 
decision  maker's  utility  can  be  represented  by  a  ratio 
utility  function  U  where 
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U(Cost, Effectiveness)  =  Effectiveness  /  Cost 

The  concept  of  indifference  curves  can  be  defined  for 
utility  functions  as  they  were  for  preference  functions. 

The  previous  discussion  of  the  ratio  preference  function, 
its  indifference  curves,  and  preference  properties  one  and 
four  hold  in  the  present  context  also.  It  was  previously 
noted  that  the  ratio  preference  function  is  equivalent  to  a 
preference  function  with  an  additive  decomposition.  The 
ratio  utility  function  is  equivalent  to  a .utility  function 
having  a  multilinear  decomposition.  That  is 

U (Cost, Ef f )  *  Eff  /  Cost 

*  ul ( Cost )  +  u2 (Ef f )  f  w  *  ul ( Cost )  *  u2 (Eff) 

where  ul  and  u2  are  single  attribute  utility  functions  of 
cost  and  effectiveness,  respectively. 

The  ratio  utility  function  is  risk  neutral  for  fixed  cost 
and  varying  effectiveness.  For  example,  with  cost  fixed  at 
3  units,  the  lottery  with  effectiveness  outcomes  6  and  12, 

<6 , 12> ,  has  expected  value  9.  The  expected  utility  of  this 
lottery  is 

(6/3  +  12/3)  /  2  =  3 

Consequently  the  certainty  equivalent  is  9  (9/3  =  3)  .  Since 
the  certainty  equivalent  equals  the  expected  value  the  risk 
neutral  case  holds. 

The  ratio  utility  function  is  risk  prone  for  fixed 
effectiveness  and  varying  cost.  For  example,  with 
effectiveness  fixed  at  4  units,  the  lottery  with  cost 
outcomes  4  and  12,  <4,12>,  has  expected  value  8.  The 
expected  utility  of  this  lottery  is 

(4/4  +  4/12)  /  2  =  2/3 

Consequently  the  certainty  equivalent  is  6  (4/6  =  2/3) . 

Since  the  certainty  equivalent,  6,  is  preferable  to  the 
expected  value,  8  (less  cost  is  better)  the  risk  prone  case 
holds. 

The  ratio  utility  function  also  exhibits  decreasing  risk 
proneness  with  increasing  cost,  again  for  fixed 
effectiveness.  This  means  that  the  difference  between  the 
certainty  equivalent  and  the  expected  value  decreased  with 
increasing  cost.  Continuing  the  last  example,  with 
effectiveness  fixed  at  4  units,  the  lottery  with  cost 
outcomes  12  and  20,  <12,20>  has  expected  value  16.  The 
expected  utility  of  this  lottery  is 

(4/12  +  4/20)  /  2  =  4/15 
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Consequently  the  certainty  equivalent  is  15.  The  certainty 
equivalent,  15,  is  preferable  to  the  expected  value,  16, 
again  indicating  risk  proneness.  The  difference  between  the 
two  values  is  now  only  1,  whereas  in  the  previous  lottery 
( <4 , 12> )  over  smaller  values  of  cost  the  difference  was  2. 
This  illustrates  decreasing  risk  proneness  with  increasing 
cost . 

For  the  ratio  utility  function  the  attributes  of  cost  and 
effectiveness  have  a  complementary  effect.  This  is 
illustrated  with  the  following  two  lotteries,  LI  and  L2. 

LI:  <(4,12)  ,  (12,4)> 

L2 :  <(4,4)  ,  (12,12)> 

Lottery  LI  yields  an  outcome  with  low  cost  and  high 
effectiveness  or  an  outcome  with  high  cost  and  low 
effectiveness.  Lottery  L2,  on  the  other  hand,  yields  an 
outcome  with  low  cost  and  low  effectiveness  or  an  outcome 
with  high  cost  and  high  effectiveness.  Lottery  LI  with 
expected  utility,  5/3,  is  preferred  to  lottery  L2  with 
expected  utility,  1.  Preference  of  LI  indicates  that  a  less 
preferred  level  on  one  attribute  cannot  be  made  up  with  a 
more  preferred  level  on  the  other  attribute.  That  is  the 
attributes  are  complementary  as  opposed  to  supplementary 
(see  page  240  of  Keaney  and  Raiffa) . 

As  was  the  situation  with  the  value  theory  application  the 
advantage  of  cost-effectiveness  ratios  is  that  the  decision 
maker  is  required  to  provide  little  or  no  information.  The 
disadvantage  is  that  the  decision  maker  must  accept  the 
unique  properties  that  this  technique  requires. 
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Appendix  C 
COEA  Data 

1.  COEA  decision  criteria.  Table  C-l  lists  a  very  general 
set  of  COEA  decision  criteria.  Refinements  of  this  set 
yielded  the  attributes  used  in  the  demonstrations. 

Table  C-l 

COEA  Decision  Criteria 

Combat  Effectiveness 
Threats 
Scenarios 
Europe 

Southwest  Asia 
Type  engagement 
Offense 
Defense 

Battlefield  Conditions 
Night  or  Day 
Weather 
Obscurants 
Attrition 

Red  systems  killed 
Blue  systems  killed 
Supportability 
Transportation 
Air 
Land 
Sea 

Maintenance 

Availability 

Reliability 

Maintainability 

Recovery 

Resupply 

Fuel 

Ammunition 
Spare  parts 
MANPRINT 
Manpower 
Personnel 
Training 
System  Safety 
Health  Hazards 
Human  Factors  Engineering 
Cost  (twenty  year  peacetime) 

Development  Costs 
Production  Costs 
Military  Construction  Costs 
Fielding  Costs 
Sustainment  Costs 


2.  COEA  combat  scenarios  descriptions.  Benefits  of 
alternatives  were  assessed  using  attrition  measures  from  a 
high  resolution  combat  simulation  model.  Three  different 
scenarios  were  represented.  In  the  three  scenarios: 

European  meeting  engagement,  European  defense,  and  Southwest 
Asian  (SWA)  meeting  engagement,  1996  blue  forces  met  2004 
red  forces.  Each  scenario  is  now  described. 

a.  European  Brigade  Meeting  Engagement.  In  this 
scenario  a  balanced  blue  brigade  meets  two  red  reinforced 
tank  battalions  in  open  terrain  beginning. at  mid-morning  of 
a  winter  day.  In  this  long  range  mounted  battle  heavy 
missile  systems  and  tanks  dominate.  Artillery  effects  are 
minimal.  Table  C-2  lists  the  weapons  systems  and  their 
numbers . 


Table  C-2' 

European  Brigade  Meeting  Engagement 
Weapons  Systems 

Blue  Red 


Weapons  System 


Tanks 

116 

56 

Infantry  Fighting 
Vehicles 

108 

38 

Dedicated  Anti- 
Tank  Systems 

24 

0 

Air  Defense 

Artillery 

17 

12 

Helicopters 

16 

10 

Other 

24 

10 

Total  Systems 

305 

126 

b.  European  Balanced  Task  Force  Defense.  In  this 
scenario  a  balanced  mechanized  infantry  task  force  defends 
against  two  red  reinforced  tank  regiments  in  open  terrain 
beginning  just  before  dawn  on  a  winter  day.  Blue  forces  arc 
presented  with  abundant  fully  exposed  moving  targets  at  long 
ranges.  Artillery  plays  a  major  role  on  both  sides.  Table 
C-3  lists  the  weapons  systems  and  their  numbers. 
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European 

Table  03 

Balanced  Task  Force 
Weapons  Systems 

Defense 

Blue 

Red 

Weapons  System 

Tanks 

28 

188 

Infantry  Fighting 
Vehicles 

30 

116 

Dedicated  Anti- 
Tank  Systems 

12 

4 

Air  Defense 
Artillery 

6 

30 

Helicopters 

16 

8 

Other 

14 

34 

Total  Systems 

106 

380 

c.  Southwest  Asian  (SWA)  brigade  meeting  engagement. 
In  this  scenario  a  blue  tank  heavy  brigade  meets  a  red 
reinforced  tank  brigade  in  open  desert  terrain  during  a 
summer  day.  Both  forces  are  moving  and  fully  exposed. 
Engagements  are  at  long  ranges  and  artillery  is  heavily 
used.  Table  04  lists  the  weapons  systems  and  their 
numbers . 
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Table  C-4 

SWA  Brigade  Meeting  Engagement 
Weapons  Systems 


Blue  Red 

Weapons  System 


Tanks 

116 

110 

Infantry  Fighting 
Vehicles 

54 

62 

Dedicated  Anti- 
Tank  Systems 

12 

12 

Air  Defense 

Artillery 

14 

15 

Helicopters 

8 

20 

Other 

52 

46 

Total  Systems 

256 

265 

3.  COEA  cost  data.  Table  C-5  gives  cost  estimates  of  the 
alternatives.  Estimates  are  in  million  FY92  constant 
dollars. 

Table  C-5 

COEA  Cost  Estimates 


Alternative 

Cost  Estimate 

Base  Case 

350 

Alt  1 

900 

Alt  2 

982 

Alt  3 

1344 

4.  COEA  loss  exchange  ratio  (LER)  data.  For  each 
alternative  and  each  scenario  statistics  below  summarize  the 
results  from  21  replications  of  a  high  resolution  combat 
model . 
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Table  C-6 

European  Brigade  Meeting  Engagement 
LER  Estimates 


Alternative 

Median 

Mean 

St . Dev 

Min 

Max 

Base  Case 

1.00 

1.06 

0.20 

0.79 

1.31 

Alt  1 

1.04 

1.08 

0.18 

0.81 

1.40 

Alt  2 

1.40 

1.45 

0.19 

1.10 

1.62 

Alt  3 

1.77 

1.81 

0.21 

1.23 

2.01 

Table  C-7 

European  Balanced  Task  Force  Defense 
LER  Estimates 


Alternative 

Median 

Mean 

St . Dev 

Min 

Max 

Base  Case 

2 .29 

2.35 

0.30 

1.99 

3 . 16 

Alt  1 

2.44 

2.46 

0.24 

2.11 

3 . 03 

Alt  2 

2.38 

2.36 

0.32 

1.87 

3.38 

Alt  3 

2.57 

2.78 

0.48 

2.15 

3 . 88 

Table  C-8 

SWA  Brigade  Meeting  Engagement 
LER  Estimates 


Alternative 

Median 

Mean 

St . Dev 

Min 

Max 

Base  Case 

1.34 

1.30 

0.24 

0.99 

1.73 

Alt  1 

1.37 

1.40 

0.19 

0.98 

1.74 

Alt  2 

1.49 

1.52 

0.18 

1.16 

1.88 

Alt  3 

1.62 

1.63 

0.17 

1.26 
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